Building texture extracting apparatus and method thereof

ABSTRACT

A building texture extracting apparatus and a method thereof are provided, wherein the building texture extracting apparatus comprises a storage unit and a processor. The storage unit is configured to store an aerial image and a panoramic image of a building. There is a coordinate correlation between the aerial image and the panoramic image. The processor defines an edge segment of a building in the aerial image, calculates an edge function according to the edge segment, projects the edge function onto the panoramic image according to the coordinate correlation to derive an edge curve function, decides an edge curve segment according to the edge curve function, captures an image area under the edge curve segment to be a building texture of the building, and stores the building texture in the storage unit

This application claims priority to Taiwan Patent Application No.100114757 filed on Apr. 28, 2011, which is hereby incorporated byreference in its entirety.

FIELD

The present invention relates to a building texture extracting apparatusand a building texture extracting method thereof. More particularly, thepresent invention relates to a building texture extracting apparatus forextracting a building texture by means of an aerial image and apanoramic image, and a building texture extracting method thereof.

BACKGROUND

In recent years, the rapid development of computer vision and computergraphics technologies has resulted in the revolutionary development ofspatial information industries worldwide, and the traditionaltwo-dimensional (2D) plane spatial information are now graduallyreplaced by the three-dimensional (3D) visualization applications and 3Dspace analysis. Accordingly, a trend of presenting a particular spatialobject by use of 3D geological information plus visualization of a real3D scene has arisen in the geological information system, andespecially, establishment of 3D building models has now become one ofthe hottest research topics.

In the process of establishing a 3D building model, one of the mostdifficult problems is how to process image data of the real 3D sceneeffectively. Generally speaking, in order to establish a complete, realand large-scale 3D building model, the image data of the real 3D scenemust be analyzed at first, then a building texture necessary forestablishing the 3D building model is identified and extracted, andfinally the extracted building texture is mapped onto the 3D buildingmodel through a mapping process to accomplish visualization of the real3D scene.

Aerial images and panoramic images are two primary kinds of image datanecessary for establishing a 3D building model. However, conventionaltechnologies cannot analyze the aerial images and panoramic imagesefficiently so as to identify and extract building textures necessaryfor establishing a 3D building model. For example, in most of theconventional technologies, a plurality of aerial images are taken atdifferent orientations and a plurality of panoramic images are taken atdifferent orientations, and then building textures of the images areprojected manually onto a 3D building model. However, this way ofprocessing makes the mapping process very complicated, time-consumingand costly; furthermore, because the building texture of each buildingis formed from a plurality of images taken at different orientations, itis difficult to determine which images shall be kept or discarded, thuscausing the building in the 3D building model to have non-uniform tones.Therefore, the conventional technologies not only fail to process theaerial images and panoramic images efficiently, but also make themapping process very complicated and necessitate beautification andcorrection of the model subsequent to the mapping process.

In view of this, an urgent need exists in the art to provide a solutionthat can correctly identify and extract a suitable building texture of abuilding by effectively analyzing an aerial image and a panoramic imageof the building so as to solve the problem of the conventionaltechnologies.

SUMMARY

An objective of the present invention is to provide a building textureextracting apparatus and a building texture extracting method thereof.The building texture extracting apparatus and the building textureextracting method thereof can effectively solve the problems with theprior art that manual extraction of the building texture makes themapping process complicated, time-consuming and costly and the problemthat the building cannot be effectively analyzed and identified to causenon-uniform tones of the building.

To achieve the aforesaid objective, the present invention provides abuilding texture extracting apparatus, which comprises a storage unitand a processor electrically connected to the storage unit. The storageunit is configured to store an aerial image and a panoramic image of abuilding, and there is a coordinate correlation between the aerial imageand the panoramic image. The processor is configured to perform thefollowing operations: defining an edge line segment of the building inthe aerial image; calculating an edge line function according to theedge line segment; calculating an edge curve function by projecting theedge line function onto the panoramic image according to the coordinatecorrelation; defining an edge curve segment of the building in thepanoramic image according to the edge curve function; capturing asub-image of the panoramic image to be a building texture of thebuilding, wherein the sub-image is under the edge curve segment; andstoring the building texture in the storage unit.

To achieve the aforesaid objective, the present invention furtherprovides a building texture extracting method for use in an electronicapparatus. The electronic apparatus comprises a storage unit and aprocessor electrically connected to the storage unit. The storage unitis configured to store an aerial image and a panoramic image of abuilding, and there is a coordinate correlation between the aerial imageand the panoramic image. The building texture extracting methodcomprises the following steps of: (a) enabling the processor to definean edge line segment of the building in the aerial image; (b) enablingthe processor to calculate an edge line function according to the edgeline segment; (c) enabling the processor to calculate an edge curvefunction by projecting the edge line function onto the panoramic imageaccording to the coordinate correlation; (d) enabling the processor todefine an edge curve segment of the building in the panoramic imageaccording to the edge curve function; (e) enabling the processor tocapture a sub-image of the panoramic image to be a building texture ofthe building, wherein the sub-image is under the edge curve segment; and(f) enabling the processor to store the building texture in the storageunit.

According to the above descriptions, the building texture extractingapparatus and the method thereof of the present invention identify andextract a suitable building texture of a building by analyzing an aerialimage and a panoramic image of the building. Through the buildingtexture extracting apparatus and the method thereof of the presentinvention, building textures of a building can be extractedautomatically, and all the building textures are extracted from the samepanoramic image. Therefore, the present invention can effectively solvethe problems with the prior art that manual extraction of the buildingtexture makes the mapping process complicated, time-consuming and costlyand the problem that the building cannot be effectively analyzed andidentified to cause non-uniform tones of the building.

The detailed technology and preferred embodiments implemented for thesubject invention are described in the following paragraphs accompanyingthe appended drawings for people skilled in this field to wellappreciate the features of the claimed invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic view depicting the internal structure andoperations of a building texture extracting apparatus 1 according to afirst embodiment of the present invention;

FIG. 2A is a schematic simulated view depicting an aerial image of anarea;

FIG. 2B depicts an edge image obtained by applying edge detection to apanoramic image correlated with the area;

FIG. 3A is a flowchart of a second embodiment; and

FIG. 3B is a flowchart of a step S24 of the second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENT

In the following descriptions, the present invention will be explainedwith reference to embodiments thereof. However, these embodiments arenot intended to limit the present invention to any specific environment,applications or particular implementations described in theseembodiments. Therefore, description of these embodiments is only forpurpose of illustration rather than to limit the present invention. Itshall be appreciated that, in the following embodiments and the attacheddrawings, elements not directly related to the present invention areomitted from depiction; and dimensional relationships among individualelements in the attached drawings are illustrated only for ease ofunderstanding but not to limit the actual scale.

A first embodiment of the present invention is a building textureextracting apparatus 1, which is configured to extract a buildingtexture of at least one building. This embodiment will be described withreference to FIG. 1, FIG. 2A and FIG. 2B together. FIG. 1 is a schematicview depicting the internal structure and operations of the buildingtexture extracting apparatus 1 of the present invention. FIG. 2A is aschematic simulated view depicting an aerial image 2 of an area, inwhich each small square represents a building. FIG. 2B depicts an edgeimage 4 obtained by applying edge detection to a panoramic imagecorrelated with the area.

Each of the aerial image 2, the panoramic image, and the edge image 4comprises an image of the at least one building whose building textureis to be extracted. In this embodiment, the buildings B1, B2, B3 and B4in the aerial image 2 are to be processed, so a panoramic imagingapparatus 5 can be placed in the space at a position corresponding tothe position P1 of FIG. 2A to acquire the panoramic image correlatedwith this area. In this way, the aerial image 2, the panoramic image,and the edge image 4 all comprise images of the buildings B1, B2, B3 andB4 to be processed.

As shown in FIG. 1, the building texture extracting apparatus 1 of thepresent invention comprises a storage unit 11 and a processor 13electrically connected to the storage unit 11. The processor 13 may beany of various processors, central processing units, microprocessors orother calculating apparatuses that are well known to those of ordinaryskill in the art. The storage unit 11 may be a memory, a floppy disk, ahard disk, a compact disk (CD), a mobile disk, a magnetic tape, adatabase accessible to networks, or any other storage media with thesame function and well known to those of ordinary skill in the art.

The storage unit 11 has the aerial image 2 and the panoramic imagestored therein. There is a coordinate correlation between the aerialimage 2 and the panoramic image, which can be obtained in various ways,for example, through a Global Positioning System (GPS). For example, theaerial image 2 can be provided by an aerial imaging apparatus 3 (e.g., asatellite or an aerostat), so the aerial image 2 has a first GPScoordinate. In this case, a GPS coordinate of each pixel of the aerialimage 2 can be derived by performing a calculation on the first GPScoordinate. On the other hand, the panoramic image can be provided by apanoramic imaging apparatus 5 (e.g., a panoramic image shootingapparatus), and by disposing the panoramic image apparatus 5 on anapparatus (e.g., a locomotive, an automobile and the like) having a GPSnavigating function, the panoramic image can have a second GPScoordinate. In this way, a GPS coordinate of each pixel of the panoramicimage can be derived by performing a calculation on the second GPScoordinate. As the aerial image 2 has the first GPS coordinate and thepanoramic image has the second GPS coordinate, the coordinatecorrelation between the aerial image 2 and the panoramic image can bedefined by the first GPS coordinate and the second GPS coordinate.

Hereinafter, how the building texture extracting apparatus 1 analyzesthe aerial image 2 and the panoramic image to identify and extract thenecessary building texture will be described with reference to FIG. 2Aand FIG. 2B.

As shown in FIG. 2A, for the buildings B1, B2, B3 and B4 in the aerialimage 2 that are to be processed, the processor 13 defines an edge linesegment (i.e., edge line segments L1, L2, L3 and L4) respectively. Eachof the edge line segments L1, L2, L3 and L4 represents a side, whichfaces towards a street, of the corresponding building. An approach todefine the edge line segments is as follows: firstly, a user marks theedge line segments L1, L2, L3 and L4 in the aerial image 2, and thenafter reading the marked aerial image 2, the processor 13 caneffectively identify and define the edge line segments L1, L2, L3 andL4. Next, the processor 13 calculates an edge line function for each ofthe edge line segments L1, L2, L3 and L4.

How to calculate the edge line function will be further described withthe edge line segment L1 as an example. Suppose that the edge linesegment L1 has three-dimensional coordinates of (x₁, y, z₁) at astarting point thereof, and three-dimensional coordinates of (x_(e), y,z_(e)) at an end point thereof. Then, the processor 13 defines a vector(Δx, 0, Δz)=(x_(e)−x₁, 0, z_(e)−z₁) according to the starting point andthe end point, and further calculates an edge line function (i.e., ƒ(x,y, z)=(x₁+Δx×t, y, z₁+Δz×t)) corresponding to the edge line segment L1in the space according to the vector, wherein the parameter y representsa height value and the parameter tε[0,1]; tε

. In other words, ƒ(x, y, z) represents coordinates of each pixel in theedge line segment L1 of the aerial image 2.

Next, according to the coordinate correlation between the aerial image 2and the panoramic image, the processor 13 uses a conventional formulafor converting line coordinates into spherical coordinates to calculatean edge curve function ƒ(μ, v) by projecting the edge line function ƒ(x,y, z) onto the panoramic image, wherein the parameter μ and theparameter v represent a coordinate of a pixel in a curve segment of thepanoramic image respectively.

Then, the processor 13 defines an edge curve segment M1 of the buildingB1 in the panoramic image according to the edge curve function ƒ(μ, v).The edge curve segment M1 can be considered to correspond to a curvesegment at an uppermost edge of the building B1, so the processor 13 cancapture a sub-image in the panoramic image to be a building texture ofthe building B1, wherein the sub-image is under the edge curve segmentM1. Hereinafter, one of the approaches to acquire the edge curve segmentM1 will be illustrated.

The processor 13 can perform edge detection on the panoramic image toobtain the edge image 4, which comprises a plurality of edge segments ofeach of the buildings. On the other hand, according to a plurality ofpreset height values and the edge curve function ƒ(μ, v) correspondingto the edge line segment L1, the processor 13 defines a plurality ofedge curve segment candidates C11, C12, C13 and C14 of the building B1that are projected onto the panoramic image and the edge image 4. Indetail, when the edge line function ƒ(x, y, z) of the edge line segmentL1 is projected onto the panoramic image 4 as the edge curve functionƒ(μ, v) of the corresponding building, different preset height values ywill be projected onto different edge curve segments C11, C12, C13 andC14. For example, the edge curve segment candidates C11, C12, C13 andC14 shown in FIG. 2B are obtained by substituting different presetheight values y1, y2, y3 and y4 into the edge line function ƒ(x, y, z)respectively and then projecting the edge line function ƒ(x, y, z) ontothe panoramic image 4 according to the edge curve function ƒ(μ, v).

Next, the processor 13 calculates an approximation between each of theedge curve segment candidates C11, C12, C13 and C14 and the edge image4, and chooses one of the edge curve segment candidates corresponding tothe largest approximation (i.e., the edge curve segment candidate C14shown in FIG. 2B) as the edge curve segment M1 of the panoramic imageand the edge image 4. Thereafter, the processor 13 captures an imagearea under the edge curve segment M1 in the panoramic image as abuilding texture that is to be extracted, and stores the buildingtexture in the storage unit 11.

Hereinafter, how the aforesaid approximation is calculated will bedescribed with reference to a preferred embodiment; however, the presentinvention is not limited to this embodiment. In detail, the aforesaidedge detection may further comprise a binarization process forconverting each pixel value of the edge image 4 into a binary value sothat each pixel of the edge image 4 only has a value of either 1 or 0.In this embodiment, a pixel value of 0 (black) in the edge image 4represents an edge of the building, and a pixel value of 1 (white)represents that the pixel is not at an edge of the building. Then, theprocessor 13 calculates an approximation for each of the edge curvesegment candidates C11, C12, C13 and C14.

Taking the edge curve segment candidate C11 as an example, the processor13 inspects pixel values of all pixels corresponding to the edge curvesegment candidate C11 in the edge image 4. When there is a pixel valueof 0, it represents that the pixel is at an edge of the building, andthe processor 13 increases the approximation by 1; and when there is apixel of 1, it represents that the pixel is not at an edge of thebuilding, and the processor 13 will not adjust the approximation. Afterthe approximations of the edge curve segment candidates C11, C12, C13and C14 have been calculated by the processor 13, the edge curve segmentcandidate corresponding to the largest approximation is taken torepresent the edge curve segment M1 (i.e., the curve segmentcorresponding to the uppermost edge of the building B1 in the panoramicimage and the edge image 4). Accordingly, the processor 13 can capture asub-image in the panoramic image to be the building texture of thebuilding B1, wherein the sub-image is under the edge curve segment M1.

Similarly, in the aforesaid way of processing, the processor 13 can alsoproject the edge line segments L2, L3 and L4 in the aerial image 2 ontothe panoramic image and the edge image 4 respectively, acquire thecorresponding edge curve segments M2, M3 and M4 respectively, andfurther capture sub-images under the edge curve segments M2, M3 and M4to be building textures of the buildings B2, B3 and B4 respectively. Inother words, the building texture extracting apparatus 1 of the presentinvention can identify and extract the building textures of thebuildings in the panoramic image by analyzing the aerial image 2 and thepanoramic image according to requirements of the user.

A second embodiment of the present invention is a building textureextracting method for use in an electronic apparatus. The electronicapparatus comprises a storage unit and a processor electricallyconnected to the storage unit. The storage unit is configured to storean aerial image and a panoramic image of a building, and there is acoordinate correlation between the aerial image and the panoramic image.The second embodiment will be described with reference to FIG. 3A andFIG. 3B together, wherein FIG. 3A is a flowchart of the secondembodiment and FIG. 3B is a detailed flowchart of step S24.

It shall be appreciated that, the electronic apparatus described in thisembodiment may be the building texture extracting apparatus 1 describedin the first embodiment, and can accomplish all the functions andoperations of the building texture extracting apparatus 1 of the firstembodiment. Furthermore, the building texture extracting methoddescribed in the second embodiment may be implemented by a computerprogram product. When the computer program product is loaded into theelectronic apparatus, a plurality of instructions comprised in thecomputer program product will be executed to accomplish the buildingtexture extracting method described in the second embodiment. Thecomputer program product may be stored in a tangible machine-readablemedium, such as a read only memory (ROM), a flash memory, a floppy disk,a hard disk, a CD, a mobile disk, a magnetic tape, a database accessibleto networks, or any other storage media with the same function and wellknown to those skilled in the art.

In detail, referring to FIG. 3A, there is shown a flowchart of thebuilding texture extracting method of the present invention. Step S21 isexecuted to enable the processor to define an edge line segment of thebuilding in the aerial image. Step S22 is executed to enable theprocessor to calculate an edge line function according to the edge linesegment. Then, step S23 is executed to enable the processor to calculatean edge curve function by projecting the edge line function onto thepanoramic image according to the coordinate correlation. Next, step S24is executed to enable the processor to define an edge curve segment ofthe building in the panoramic image according to the edge curvefunction. Step S25 is executed to enable the processor to capture asub-image under the edge curve segment to be a building texture of thebuilding. Finally, step S26 is executed to enable the processor to storethe building texture in the storage unit.

Further speaking, as shown in FIG. 3B, the step S24 comprises thefollowing sub-steps. Firstly, sub-step S241 is executed to enable theprocessor to define a plurality of edge curve segment candidates of thebuilding that are projected onto the panoramic image according to aplurality of preset height values of the building and the edge curvefunction. Then, sub-step S242 is executed to enable the processor toderive an edge image by applying edge detection to the panoramic image.Sub-step S243 is executed to enable the processor to calculate anapproximation between each of the edge curve segment candidates and theedge image. Finally, sub-step S244 is executed to enable the processorto choose one of the edge curve segment candidates that has the largestapproximation as the edge curve segment of the building in the panoramicimage.

It shall be appreciated that, the coordinate correlation between theaerial image and the panoramic image can be defined by the GPS.Specifically, if the aerial image has a first GPS coordinate and thepanoramic image has a second GPS coordinate, then the coordinatecorrelation can be defined by the first GPS coordinate and the secondGPS coordinate. Further, the edge detection described in the sub-stepS242 may comprise a binarization process so that, after the edgedetection, each of a plurality of pixels comprised in the edge segmentsof the building has a binary value. Furthermore, in addition to theaforesaid steps, the second embodiment can also execute all theoperations and functions set forth in the first embodiment. How thesecond embodiment executes these operations and functions will bereadily appreciated by those of ordinary skill in the art based on theexplanation of the first embodiment, and thus will not be furtherdescribed herein.

According to the above descriptions, the building texture extractingapparatus and the method thereof of the present invention identify andextract a suitable building texture of at least one building byanalyzing an aerial image and a panoramic image that comprise the atleast one building. Through the building texture extracting apparatusand the method thereof of the present invention, building textures of abuilding can be extracted automatically, and all the building texturesare extracted from the same panoramic image. Therefore, the presentinvention can effectively solve the problems with the prior art thatmanual extraction of the building texture makes the mapping processcomplicated, time-consuming and costly and the problem that the buildingcannot be effectively analyzed and identified to cause non-uniform tonesof the building.

The above disclosure is related to the detailed technical contents andinventive features thereof. People skilled in this field may proceedwith a variety of modifications and replacements based on thedisclosures and suggestions of the invention as described withoutdeparting from the characteristics thereof. Nevertheless, although suchmodifications and replacements are not fully disclosed in the abovedescriptions, they have substantially been covered in the followingclaims as appended.

1. A building texture extracting apparatus, comprising: a storage unit,being configured to store an aerial image and a panoramic image of abuilding, there being a coordinate correlation between the aerial imageand the panoramic image; and a processor, being electrically connectedto the storage unit and configured to perform the following operations:defining an edge line segment of the building in the aerial image,calculating an edge line function according to the edge line segment,calculating an edge curve function by projecting the edge line functiononto the panoramic image according to the coordinate correlation,defining an edge curve segment of the building in the panoramic imageaccording to the edge curve function, capturing a sub-image of thepanoramic image to be a building texture of the building, the sub-imagebeing under the edge curve segment, and storing the building texture inthe storage unit.
 2. The building texture extracting apparatus asclaimed in claim 1, wherein the processor performs the followingoperations to define the edge curve segment of the building in thepanoramic image: defining a plurality of edge curve segment candidatesof the building that are projected onto the panoramic image according aplurality of preset height values of the building and the edge curvefunction; deriving an edge image by applying edge detection to thepanoramic image; calculating an approximation between each of the edgecurve segment candidates and the edge image; and choosing one of theedge curve segment candidates that has the largest approximation as theedge curve segment of the building in the panoramic image.
 3. Thebuilding texture extracting apparatus as claimed in claim 2, wherein theedge detection is utilized to obtain a plurality of edge segments of thebuilding in the panoramic image.
 4. The building texture extractingapparatus as claimed in claim 3, wherein the edge detection comprises abinarization process so that each of a plurality of pixels comprised inthe edge segments of the building is a binary value.
 5. The buildingtexture extracting apparatus as claimed in claim 1, wherein the aerialimage has a first Global Positioning System (GPS) coordinate, thepanoramic image has a second GPS coordinate, and the first GPScoordinate and the second GPS coordinate define the coordinatecorrelation.
 6. A building texture extracting method for use in anelectronic apparatus, the electronic apparatus comprising a storage unitand a processor electrically connected to the storage unit, the storageunit being configured to store an aerial image and a panoramic image ofa building, there being a coordinate correlation between the aerialimage and the panoramic image, the building texture extracting methodcomprising the following steps of: (a) enabling the processor to definean edge line segment of the building in the aerial image; (b) enablingthe processor to calculate an edge line function according to the edgeline segment; (c) enabling the processor to calculate an edge curvefunction by projecting the edge line function onto the panoramic imageaccording to the coordinate correlation; (d) enabling the processor todefine an edge curve segment of the building in the panoramic imageaccording to the edge curve function; (e) enabling the processor tocapture a sub-image of the panoramic image to be a building texture ofthe building, the sub-image being under the edge curve segment; and (f)enabling the processor to store the building texture in the storageunit.
 7. The building texture extracting method as claimed in claim 6,wherein the step (d) further comprises the following sub-steps of: (d1)enabling the processor to define a plurality of edge curve segmentcandidates of the building that are projected onto the panoramic imageaccording a plurality of preset height values of the building and theedge curve function; (d2) enabling the processor to derive an edge imageby applying edge detection to the panoramic image; (d3) enabling theprocessor to calculate an approximation between each of the edge curvesegment candidates and the edge image; and (d4) enabling the processorto choose one of the edge curve segment candidates that has the largestapproximation as the edge curve segment of the building in the panoramicimage.
 8. The building texture extracting method as claimed in claim 7,wherein the edge detection is utilized to obtain a plurality of edgesegments of the building in the panoramic image.
 9. The building textureextracting method as claimed in claim 8, wherein the edge detectioncomprises a binarization process so that each of a plurality of pixelscomprised in the edge segments of the building is a binary value. 10.The building texture extracting method as claimed in claim 6, whereinthe aerial image has a first GPS coordinate, the panoramic image has asecond GPS coordinate, and the first GPS coordinate and the second GPScoordinate define the coordinate correlation.